Point and Interval Estimation

Last week, we talked about estimating population proportions.
Our point estimates were proportions of a random sample, drawn with replacement, from the population.
To we’d be, we used interval estimates. In particular, calibrated interval estimates. Confidence intervals.
- To get these intervals, we add ‘arms’ of equal length to our point estimate.
- We choose the arm length so that, 95% of the time, the population proportion is within them.

Calibration of Interval Estimates

We can do that using the sampling distribution of our point estimate.
We’d draw arms out from the sampling distribution’s until they span 95% of point estimates.
We can see that this is the right length by a shift of perspective.
- A point estimate can touch the mean with its arms iff¹ the mean can touch it with equally long arms.
- So 95% of intervals cover if and only if 95% of point estimates are within the mean’s arms.
In practice, we can’t do exactly that. We don’t know the sampling distribution.
But we can use an estimate of the sampling distribution in its place.
If it’s a good one, we’ll get approximately 95% coverage.

A Parametric Estimate of the Sampling Distribution

Our point estimate and its sampling distribution

Our estimate of its sampling distribution

It helps to know the parametric form of the sampling distribution: it’s binomial.
- It’s the distribution of the proportion heads in 625 flips of a coin with probability $p \approx 0.70$ of heads.
- where $p \approx 0.70$ is the proportion of registered voters who will vote.
By parametric form, I mean a formula in terms of some parameters.
- That tells us what parameters we need to estimate to estimate the sampling distribution.
- In this case, the only (unknown) parameter is the probability of heads, $p$.

A Parametric Estimate of the Sampling Distribution

Our point estimate and its sampling distribution

Our estimate of its sampling distribution

In R, dbinom(x, n, p), calculates this formula where …
- x is the number of heads
- n is the number of flips
- p is the probability of heads, i.e., the proportion $\bar y$ of registered voters who will vote.
To estimate the sampling distribution, we plug in our point estimate $\bar Y$ for $p$.
There’s another way to think about this that generalizes to more complicated situations.

The Bootstrap Interpretation

The estimated sampling distribution is $\text{Binomial}(n,p)/n$ for $n=625$ and $p=\bar Y\approx 0.68$.
- It’s the distribution of the proportion heads in 625 flips of a coin with probability $p \approx 0.68$ of heads.
- where $p \approx 0.68$ is the proportion of voters we’ve polled who will vote.
That is, it’s the sampling distribution of a ‘poll’ of the people in our sample, i.e.
- roll a 625-sided die 625 times
- call up the corresponding person in our sample
- and counting up the yeses we hear
Note that this is random because we’re drawing with replacement.
- Each time we run this poll, we call each person in our sample 0,1,2,… times
- And the number of times we call them is random.

The Bootstrap Interpretation

If we plot our voters on a map, you can see the idea in visual terms.
- On the left, we have the population.
- In the middle, we have our sample. It’s drawn, with replacement, from the population.
- On the right, we have something else. A new sample drawn, with replacement, from the sample.
Each ‘call’ that a person receives increases the size of their dot: $\text{circle area} \propto \text{number of calls}$.
In the sample, even though it’s drawn with replacement, all dots are the same size.
- Because we draw from such a large population, nobody gets called twice.
In the bootstrap sample, dots vary in size.
- Because we draw $n$ people from a sample of size $n$, it’s almost impossible not to call somebody twice.

Bootstrapping

Before the election, we don’t observe the population. But we do observe the sample.
- And we can sample from the sample act as if it were the population.
- We’ll take repeated random samples of size 625, with replacement, from our sample of size 625.
We call these bootstrap samples and estimates based on them bootstrap estimates.
- The distribution of these estimates is called the bootstrap sampling distribution.
- If the sample is like the population, this should be like our estimator’s actual sampling distribution.

The Sample \[ \begin{array}{r|rrrr|r} i & 1 & 2 & \dots & 625 & \bar{Y}_{625} \\ Y_i & 1 & 1 & \dots & 1 & 0.68 \\ \end{array} \]

The Bootstrap Sample

\[ \begin{array}{r|rrrr|r} i & 1 & 2 & \dots & 625 & \bar{Y}_{625}^* \\ Y_i^* & 1 & 1 & \dots & 1 & 0.63 \\ \end{array} \]

The Population

\[ \begin{array}{r|rrrr|r} j & 1 & 2 & \dots & 7.23M & \bar{y}_{7.23M} \\ y_{j} & 1 & 1 & \dots & 1 & 0.70 \\ \end{array} \]

The ‘Bootstrap Population’ — The Sample \[ \begin{array}{r|rrrr|r} j & 1 & 2 & \dots & 625 & \bar{y}^*_{625} \\ y_j^* & 1 & 1 & \dots & 1 & 0.68 \\ \end{array} \]

The Bootstrap is Nonparametric

bootstrap.samples = array(dim=10000)
for(rr in 1:10000) {
    Y.star = Y[sample(1:n, n, replace=TRUE)]
    bootstrap.samples[rr] = mean(Y.star) 
}

We do not need to know the parametric form of our sampling distribution to use it.
All we do is re-run our poll acting as if our sample were the population.
- We can do this no matter what we’re estimating.
- So let’s try it out on some other stuff.

Comparing Black and Non-Black Turnout

\[ \small{ \begin{array}{r|rr|rr|r|rr|rrrrr} \text{call} & 1 & & 2 & & \dots & 625 & & & & & & \\ \text{question} & X_1 & Y_1 & X_2 & Y_2 & \dots & X_{625} & Y_{625} & \overline{X}_{625} & \overline{Y}_{625} &\frac{\sum_{i:X_i=0} Y_i}{\sum_{i:X_i=0} 1} & \frac{\sum_{i:X_i=1} Y_i}{\sum_{i:X_i=1} 1} & \text{difference} \\ \text{outcome} & \underset{\textcolor{gray}{x_{869369}}}{0} & \underset{\textcolor{gray}{y_{869369}}}{1} & \underset{\textcolor{gray}{x_{4428455}}}{1} & \underset{\textcolor{gray}{y_{4428455}}}{1} & \dots & \underset{\textcolor{gray}{x_{1268868}}}{0} & \underset{\textcolor{gray}{y_{1268868}}}{1} & 0.28 & 0.68 & 0.68 & 0.69 & 0.01 \\ \end{array} } \]

It’s useful to predict turnout among Black voters because they tend to vote differently than non-Black voters.
Let’s suppose that we’re interested in the difference in turnout between Black and non-Black voters.
This is a bit reductive, but it’s the beginning of the semester, so we’re keeping things simple.
We know from the voter file that 30% of registered voters are Black.
Let’s suppose we asked the people we polled if they were Black, recording their answer in a covariate $X_i$.

A Covariate? What’s That?

Comparing Black and Non-Black Turnout

\[ \small{ \begin{array}{r|rr|rr|r|rr|rrrrr} \text{call} & 1 & & 2 & & \dots & 625 & & & & & & \\ \text{question} & X_1 & Y_1 & X_2 & Y_2 & \dots & X_{625} & Y_{625} & \overline{X}_{625} & \overline{Y}_{625} &\frac{\sum_{i:X_i=0} Y_i}{\sum_{i:X_i=0} 1} & \frac{\sum_{i:X_i=1} Y_i}{\sum_{i:X_i=1} 1} & \text{difference} \\ \text{outcome} & \underset{\textcolor{gray}{x_{869369}}}{0} & \underset{\textcolor{gray}{y_{869369}}}{1} & \underset{\textcolor{gray}{x_{4428455}}}{1} & \underset{\textcolor{gray}{y_{4428455}}}{1} & \dots & \underset{\textcolor{gray}{x_{1268868}}}{0} & \underset{\textcolor{gray}{y_{1268868}}}{1} & 0.28 & 0.68 & 0.68 & 0.69 & 0.01 \\ \end{array} } \]

In our poll, we found that…
- 172 of the people we called (28%) were Black with a turnout rate of 0.69
- 453 of the people we called (72%) were non-Black with a turnout rate of 0.68
After the election, we found that …
- Turnout among Black registered voters was 0.74
- Turnout among non-Black registered voters was 0.68.
We’d estimate the difference to be $0.69-0.68 \approx 0.01$.
The actual difference was $0.74-0.68 \approx 0.06$.
Depending on what you’re doing, that point estimate might be good enough or might not be.
It’d be nice to have an interval estimate. So let’s think about this difference’s sampling distribution.

The Turnout Difference’s Sampling Distribution

difference.samples = array(dim=10000)
for(rr in 1:10000) {
    I = sample(1:m, n, replace=TRUE)
    X = x[I]
    Y = y[I]
    difference.samples[rr] = mean(Y[X==1]) - mean(Y[X==0])
}

Here’s what the sampling distribution of this difference in turnout looks like.
As before, if we can estimate it we can use that estimate to get a 95% confidence interval.
But, unlike before, we don’t really know the parametric form of its sampling distribution.
Or—at least—it’d be a pain to work it out. So we’ll use the bootstrap to estimate it.

	call 1		call 2		call 3		call 4		call 5		call 6		call 7		call 8		call 9		call 10		call 11		call 12		call 13		call 14		call 15		call 16		call 17		call 18		call 19		call 20		call 21		call 22		call 23		call 24		call 25		call 26		call 27		call 28		call 29		call 30		call 31		call 32		call 33		call 34		call 35		call 36		call 37		call 38		call 39		call 40		call 41		call 42		call 43		call 44		call 45		call 46		call 47		call 48		call 49		call 50		call 51		call 52		call 53		call 54		call 55		call 56		call 57		call 58		call 59		call 60		call 61		call 62		call 63		call 64		call 65		call 66		call 67		call 68		call 69		call 70		call 71		call 72		call 73		call 74		call 75		call 76		call 77		call 78		call 79		call 80		call 81		call 82		call 83		call 84		call 85		call 86		call 87		call 88		call 89		call 90		call 91		call 92		call 93		call 94		call 95		call 96		call 97		call 98		call 99		call 100		call 101		call 102		call 103		call 104		call 105		call 106		call 107		call 108		call 109		call 110		call 111		call 112		call 113		call 114		call 115		call 116		call 117		call 118		call 119		call 120		call 121		call 122		call 123		call 124		call 125		call 126		call 127		call 128		call 129		call 130		call 131		call 132		call 133		call 134		call 135		call 136		call 137		call 138		call 139		call 140		call 141		call 142		call 143		call 144		call 145		call 146		call 147		call 148		call 149		call 150		call 151		call 152		call 153		call 154		call 155		call 156		call 157		call 158		call 159		call 160		call 161		call 162		call 163		call 164		call 165		call 166		call 167		call 168		call 169		call 170		call 171		call 172		call 173		call 174		call 175		call 176		call 177		call 178		call 179		call 180		call 181		call 182		call 183		call 184		call 185		call 186		call 187		call 188		call 189		call 190		call 191		call 192		call 193		call 194		call 195		call 196		call 197		call 198		call 199		call 200		call 201		call 202		call 203		call 204		call 205		call 206		call 207		call 208		call 209		call 210		call 211		call 212		call 213		call 214		call 215		call 216		call 217		call 218		call 219		call 220		call 221		call 222		call 223		call 224		call 225		call 226		call 227		call 228		call 229		call 230		call 231		call 232		call 233		call 234		call 235		call 236		call 237		call 238		call 239		call 240		call 241		call 242		call 243		call 244		call 245		call 246		call 247		call 248		call 249		call 250		call 251		call 252		call 253		call 254		call 255		call 256		call 257		call 258		call 259		call 260		call 261		call 262		call 263		call 264		call 265		call 266		call 267		call 268		call 269		call 270		call 271		call 272		call 273		call 274		call 275		call 276		call 277		call 278		call 279		call 280		call 281		call 282		call 283		call 284		call 285		call 286		call 287		call 288		call 289		call 290		call 291		call 292		call 293		call 294		call 295		call 296		call 297		call 298		call 299		call 300		call 301		call 302		call 303		call 304		call 305		call 306		call 307		call 308		call 309		call 310		call 311		call 312		call 313		call 314		call 315		call 316		call 317		call 318		call 319		call 320		call 321		call 322		call 323		call 324		call 325		call 326		call 327		call 328		call 329		call 330		call 331		call 332		call 333		call 334		call 335		call 336		call 337		call 338		call 339		call 340		call 341		call 342		call 343		call 344		call 345		call 346		call 347		call 348		call 349		call 350		call 351		call 352		call 353		call 354		call 355		call 356		call 357		call 358		call 359		call 360		call 361		call 362		call 363		call 364		call 365		call 366		call 367		call 368		call 369		call 370		call 371		call 372		call 373		call 374		call 375		call 376		call 377		call 378		call 379		call 380		call 381		call 382		call 383		call 384		call 385		call 386		call 387		call 388		call 389		call 390		call 391		call 392		call 393		call 394		call 395		call 396		call 397		call 398		call 399		call 400		call 401		call 402		call 403		call 404		call 405		call 406		call 407		call 408		call 409		call 410		call 411		call 412		call 413		call 414		call 415		call 416		call 417		call 418		call 419		call 420		call 421		call 422		call 423		call 424		call 425		call 426		call 427		call 428		call 429		call 430		call 431		call 432		call 433		call 434		call 435		call 436		call 437		call 438		call 439		call 440		call 441		call 442		call 443		call 444		call 445		call 446		call 447		call 448		call 449		call 450		call 451		call 452		call 453		call 454		call 455		call 456		call 457		call 458		call 459		call 460		call 461		call 462		call 463		call 464		call 465		call 466		call 467		call 468		call 469		call 470		call 471		call 472		call 473		call 474		call 475		call 476		call 477		call 478		call 479		call 480		call 481		call 482		call 483		call 484		call 485		call 486		call 487		call 488		call 489		call 490		call 491		call 492		call 493		call 494		call 495		call 496		call 497		call 498		call 499		call 500		call 501		call 502		call 503		call 504		call 505		call 506		call 507		call 508		call 509		call 510		call 511		call 512		call 513		call 514		call 515		call 516		call 517		call 518		call 519		call 520		call 521		call 522		call 523		call 524		call 525		call 526		call 527		call 528		call 529		call 530		call 531		call 532		call 533		call 534		call 535		call 536		call 537		call 538		call 539		call 540		call 541		call 542		call 543		call 544		call 545		call 546		call 547		call 548		call 549		call 550		call 551		call 552		call 553		call 554		call 555		call 556		call 557		call 558		call 559		call 560		call 561		call 562		call 563		call 564		call 565		call 566		call 567		call 568		call 569		call 570		call 571		call 572		call 573		call 574		call 575		call 576		call 577		call 578		call 579		call 580		call 581		call 582		call 583		call 584		call 585		call 586		call 587		call 588		call 589		call 590		call 591		call 592		call 593		call 594		call 595		call 596		call 597		call 598		call 599		call 600		call 601		call 602		call 603		call 604		call 605		call 606		call 607		call 608		call 609		call 610		call 611		call 612		call 613		call 614		call 615		call 616		call 617		call 618		call 619		call 620		call 621		call 622		call 623		call 624		call 625
Poll	$X_{1}$	$Y_{1}$	$X_{2}$	$Y_{2}$	$X_{3}$	$Y_{3}$	$X_{4}$	$Y_{4}$	$X_{5}$	$Y_{5}$	$X_{6}$	$Y_{6}$	$X_{7}$	$Y_{7}$	$X_{8}$	$Y_{8}$	$X_{9}$	$Y_{9}$	$X_{10}$	$Y_{10}$	$X_{11}$	$Y_{11}$	$X_{12}$	$Y_{12}$	$X_{13}$	$Y_{13}$	$X_{14}$	$Y_{14}$	$X_{15}$	$Y_{15}$	$X_{16}$	$Y_{16}$	$X_{17}$	$Y_{17}$	$X_{18}$	$Y_{18}$	$X_{19}$	$Y_{19}$	$X_{20}$	$Y_{20}$	$X_{21}$	$Y_{21}$	$X_{22}$	$Y_{22}$	$X_{23}$	$Y_{23}$	$X_{24}$	$Y_{24}$	$X_{25}$	$Y_{25}$	$X_{26}$	$Y_{26}$	$X_{27}$	$Y_{27}$	$X_{28}$	$Y_{28}$	$X_{29}$	$Y_{29}$	$X_{30}$	$Y_{30}$	$X_{31}$	$Y_{31}$	$X_{32}$	$Y_{32}$	$X_{33}$	$Y_{33}$	$X_{34}$	$Y_{34}$	$X_{35}$	$Y_{35}$	$X_{36}$	$Y_{36}$	$X_{37}$	$Y_{37}$	$X_{38}$	$Y_{38}$	$X_{39}$	$Y_{39}$	$X_{40}$	$Y_{40}$	$X_{41}$	$Y_{41}$	$X_{42}$	$Y_{42}$	$X_{43}$	$Y_{43}$	$X_{44}$	$Y_{44}$	$X_{45}$	$Y_{45}$	$X_{46}$	$Y_{46}$	$X_{47}$	$Y_{47}$	$X_{48}$	$Y_{48}$	$X_{49}$	$Y_{49}$	$X_{50}$	$Y_{50}$	$X_{51}$	$Y_{51}$	$X_{52}$	$Y_{52}$	$X_{53}$	$Y_{53}$	$X_{54}$	$Y_{54}$	$X_{55}$	$Y_{55}$	$X_{56}$	$Y_{56}$	$X_{57}$	$Y_{57}$	$X_{58}$	$Y_{58}$	$X_{59}$	$Y_{59}$	$X_{60}$	$Y_{60}$	$X_{61}$	$Y_{61}$	$X_{62}$	$Y_{62}$	$X_{63}$	$Y_{63}$	$X_{64}$	$Y_{64}$	$X_{65}$	$Y_{65}$	$X_{66}$	$Y_{66}$	$X_{67}$	$Y_{67}$	$X_{68}$	$Y_{68}$	$X_{69}$	$Y_{69}$	$X_{70}$	$Y_{70}$	$X_{71}$	$Y_{71}$	$X_{72}$	$Y_{72}$	$X_{73}$	$Y_{73}$	$X_{74}$	$Y_{74}$	$X_{75}$	$Y_{75}$	$X_{76}$	$Y_{76}$	$X_{77}$	$Y_{77}$	$X_{78}$	$Y_{78}$	$X_{79}$	$Y_{79}$	$X_{80}$	$Y_{80}$	$X_{81}$	$Y_{81}$	$X_{82}$	$Y_{82}$	$X_{83}$	$Y_{83}$	$X_{84}$	$Y_{84}$	$X_{85}$	$Y_{85}$	$X_{86}$	$Y_{86}$	$X_{87}$	$Y_{87}$	$X_{88}$	$Y_{88}$	$X_{89}$	$Y_{89}$	$X_{90}$	$Y_{90}$	$X_{91}$	$Y_{91}$	$X_{92}$	$Y_{92}$	$X_{93}$	$Y_{93}$	$X_{94}$	$Y_{94}$	$X_{95}$	$Y_{95}$	$X_{96}$	$Y_{96}$	$X_{97}$	$Y_{97}$	$X_{98}$	$Y_{98}$	$X_{99}$	$Y_{99}$	$X_{100}$	$Y_{100}$	$X_{101}$	$Y_{101}$	$X_{102}$	$Y_{102}$	$X_{103}$	$Y_{103}$	$X_{104}$	$Y_{104}$	$X_{105}$	$Y_{105}$	$X_{106}$	$Y_{106}$	$X_{107}$	$Y_{107}$	$X_{108}$	$Y_{108}$	$X_{109}$	$Y_{109}$	$X_{110}$	$Y_{110}$	$X_{111}$	$Y_{111}$	$X_{112}$	$Y_{112}$	$X_{113}$	$Y_{113}$	$X_{114}$	$Y_{114}$	$X_{115}$	$Y_{115}$	$X_{116}$	$Y_{116}$	$X_{117}$	$Y_{117}$	$X_{118}$	$Y_{118}$	$X_{119}$	$Y_{119}$	$X_{120}$	$Y_{120}$	$X_{121}$	$Y_{121}$	$X_{122}$	$Y_{122}$	$X_{123}$	$Y_{123}$	$X_{124}$	$Y_{124}$	$X_{125}$	$Y_{125}$	$X_{126}$	$Y_{126}$	$X_{127}$	$Y_{127}$	$X_{128}$	$Y_{128}$	$X_{129}$	$Y_{129}$	$X_{130}$	$Y_{130}$	$X_{131}$	$Y_{131}$	$X_{132}$	$Y_{132}$	$X_{133}$	$Y_{133}$	$X_{134}$	$Y_{134}$	$X_{135}$	$Y_{135}$	$X_{136}$	$Y_{136}$	$X_{137}$	$Y_{137}$	$X_{138}$	$Y_{138}$	$X_{139}$	$Y_{139}$	$X_{140}$	$Y_{140}$	$X_{141}$	$Y_{141}$	$X_{142}$	$Y_{142}$	$X_{143}$	$Y_{143}$	$X_{144}$	$Y_{144}$	$X_{145}$	$Y_{145}$	$X_{146}$	$Y_{146}$	$X_{147}$	$Y_{147}$	$X_{148}$	$Y_{148}$	$X_{149}$	$Y_{149}$	$X_{150}$	$Y_{150}$	$X_{151}$	$Y_{151}$	$X_{152}$	$Y_{152}$	$X_{153}$	$Y_{153}$	$X_{154}$	$Y_{154}$	$X_{155}$	$Y_{155}$	$X_{156}$	$Y_{156}$	$X_{157}$	$Y_{157}$	$X_{158}$	$Y_{158}$	$X_{159}$	$Y_{159}$	$X_{160}$	$Y_{160}$	$X_{161}$	$Y_{161}$	$X_{162}$	$Y_{162}$	$X_{163}$	$Y_{163}$	$X_{164}$	$Y_{164}$	$X_{165}$	$Y_{165}$	$X_{166}$	$Y_{166}$	$X_{167}$	$Y_{167}$	$X_{168}$	$Y_{168}$	$X_{169}$	$Y_{169}$	$X_{170}$	$Y_{170}$	$X_{171}$	$Y_{171}$	$X_{172}$	$Y_{172}$	$X_{173}$	$Y_{173}$	$X_{174}$	$Y_{174}$	$X_{175}$	$Y_{175}$	$X_{176}$	$Y_{176}$	$X_{177}$	$Y_{177}$	$X_{178}$	$Y_{178}$	$X_{179}$	$Y_{179}$	$X_{180}$	$Y_{180}$	$X_{181}$	$Y_{181}$	$X_{182}$	$Y_{182}$	$X_{183}$	$Y_{183}$	$X_{184}$	$Y_{184}$	$X_{185}$	$Y_{185}$	$X_{186}$	$Y_{186}$	$X_{187}$	$Y_{187}$	$X_{188}$	$Y_{188}$	$X_{189}$	$Y_{189}$	$X_{190}$	$Y_{190}$	$X_{191}$	$Y_{191}$	$X_{192}$	$Y_{192}$	$X_{193}$	$Y_{193}$	$X_{194}$	$Y_{194}$	$X_{195}$	$Y_{195}$	$X_{196}$	$Y_{196}$	$X_{197}$	$Y_{197}$	$X_{198}$	$Y_{198}$	$X_{199}$	$Y_{199}$	$X_{200}$	$Y_{200}$	$X_{201}$	$Y_{201}$	$X_{202}$	$Y_{202}$	$X_{203}$	$Y_{203}$	$X_{204}$	$Y_{204}$	$X_{205}$	$Y_{205}$	$X_{206}$	$Y_{206}$	$X_{207}$	$Y_{207}$	$X_{208}$	$Y_{208}$	$X_{209}$	$Y_{209}$	$X_{210}$	$Y_{210}$	$X_{211}$	$Y_{211}$	$X_{212}$	$Y_{212}$	$X_{213}$	$Y_{213}$	$X_{214}$	$Y_{214}$	$X_{215}$	$Y_{215}$	$X_{216}$	$Y_{216}$	$X_{217}$	$Y_{217}$	$X_{218}$	$Y_{218}$	$X_{219}$	$Y_{219}$	$X_{220}$	$Y_{220}$	$X_{221}$	$Y_{221}$	$X_{222}$	$Y_{222}$	$X_{223}$	$Y_{223}$	$X_{224}$	$Y_{224}$	$X_{225}$	$Y_{225}$	$X_{226}$	$Y_{226}$	$X_{227}$	$Y_{227}$	$X_{228}$	$Y_{228}$	$X_{229}$	$Y_{229}$	$X_{230}$	$Y_{230}$	$X_{231}$	$Y_{231}$	$X_{232}$	$Y_{232}$	$X_{233}$	$Y_{233}$	$X_{234}$	$Y_{234}$	$X_{235}$	$Y_{235}$	$X_{236}$	$Y_{236}$	$X_{237}$	$Y_{237}$	$X_{238}$	$Y_{238}$	$X_{239}$	$Y_{239}$	$X_{240}$	$Y_{240}$	$X_{241}$	$Y_{241}$	$X_{242}$	$Y_{242}$	$X_{243}$	$Y_{243}$	$X_{244}$	$Y_{244}$	$X_{245}$	$Y_{245}$	$X_{246}$	$Y_{246}$	$X_{247}$	$Y_{247}$	$X_{248}$	$Y_{248}$	$X_{249}$	$Y_{249}$	$X_{250}$	$Y_{250}$	$X_{251}$	$Y_{251}$	$X_{252}$	$Y_{252}$	$X_{253}$	$Y_{253}$	$X_{254}$	$Y_{254}$	$X_{255}$	$Y_{255}$	$X_{256}$	$Y_{256}$	$X_{257}$	$Y_{257}$	$X_{258}$	$Y_{258}$	$X_{259}$	$Y_{259}$	$X_{260}$	$Y_{260}$	$X_{261}$	$Y_{261}$	$X_{262}$	$Y_{262}$	$X_{263}$	$Y_{263}$	$X_{264}$	$Y_{264}$	$X_{265}$	$Y_{265}$	$X_{266}$	$Y_{266}$	$X_{267}$	$Y_{267}$	$X_{268}$	$Y_{268}$	$X_{269}$	$Y_{269}$	$X_{270}$	$Y_{270}$	$X_{271}$	$Y_{271}$	$X_{272}$	$Y_{272}$	$X_{273}$	$Y_{273}$	$X_{274}$	$Y_{274}$	$X_{275}$	$Y_{275}$	$X_{276}$	$Y_{276}$	$X_{277}$	$Y_{277}$	$X_{278}$	$Y_{278}$	$X_{279}$	$Y_{279}$	$X_{280}$	$Y_{280}$	$X_{281}$	$Y_{281}$	$X_{282}$	$Y_{282}$	$X_{283}$	$Y_{283}$	$X_{284}$	$Y_{284}$	$X_{285}$	$Y_{285}$	$X_{286}$	$Y_{286}$	$X_{287}$	$Y_{287}$	$X_{288}$	$Y_{288}$	$X_{289}$	$Y_{289}$	$X_{290}$	$Y_{290}$	$X_{291}$	$Y_{291}$	$X_{292}$	$Y_{292}$	$X_{293}$	$Y_{293}$	$X_{294}$	$Y_{294}$	$X_{295}$	$Y_{295}$	$X_{296}$	$Y_{296}$	$X_{297}$	$Y_{297}$	$X_{298}$	$Y_{298}$	$X_{299}$	$Y_{299}$	$X_{300}$	$Y_{300}$	$X_{301}$	$Y_{301}$	$X_{302}$	$Y_{302}$	$X_{303}$	$Y_{303}$	$X_{304}$	$Y_{304}$	$X_{305}$	$Y_{305}$	$X_{306}$	$Y_{306}$	$X_{307}$	$Y_{307}$	$X_{308}$	$Y_{308}$	$X_{309}$	$Y_{309}$	$X_{310}$	$Y_{310}$	$X_{311}$	$Y_{311}$	$X_{312}$	$Y_{312}$	$X_{313}$	$Y_{313}$	$X_{314}$	$Y_{314}$	$X_{315}$	$Y_{315}$	$X_{316}$	$Y_{316}$	$X_{317}$	$Y_{317}$	$X_{318}$	$Y_{318}$	$X_{319}$	$Y_{319}$	$X_{320}$	$Y_{320}$	$X_{321}$	$Y_{321}$	$X_{322}$	$Y_{322}$	$X_{323}$	$Y_{323}$	$X_{324}$	$Y_{324}$	$X_{325}$	$Y_{325}$	$X_{326}$	$Y_{326}$	$X_{327}$	$Y_{327}$	$X_{328}$	$Y_{328}$	$X_{329}$	$Y_{329}$	$X_{330}$	$Y_{330}$	$X_{331}$	$Y_{331}$	$X_{332}$	$Y_{332}$	$X_{333}$	$Y_{333}$	$X_{334}$	$Y_{334}$	$X_{335}$	$Y_{335}$	$X_{336}$	$Y_{336}$	$X_{337}$	$Y_{337}$	$X_{338}$	$Y_{338}$	$X_{339}$	$Y_{339}$	$X_{340}$	$Y_{340}$	$X_{341}$	$Y_{341}$	$X_{342}$	$Y_{342}$	$X_{343}$	$Y_{343}$	$X_{344}$	$Y_{344}$	$X_{345}$	$Y_{345}$	$X_{346}$	$Y_{346}$	$X_{347}$	$Y_{347}$	$X_{348}$	$Y_{348}$	$X_{349}$	$Y_{349}$	$X_{350}$	$Y_{350}$	$X_{351}$	$Y_{351}$	$X_{352}$	$Y_{352}$	$X_{353}$	$Y_{353}$	$X_{354}$	$Y_{354}$	$X_{355}$	$Y_{355}$	$X_{356}$	$Y_{356}$	$X_{357}$	$Y_{357}$	$X_{358}$	$Y_{358}$	$X_{359}$	$Y_{359}$	$X_{360}$	$Y_{360}$	$X_{361}$	$Y_{361}$	$X_{362}$	$Y_{362}$	$X_{363}$	$Y_{363}$	$X_{364}$	$Y_{364}$	$X_{365}$	$Y_{365}$	$X_{366}$	$Y_{366}$	$X_{367}$	$Y_{367}$	$X_{368}$	$Y_{368}$	$X_{369}$	$Y_{369}$	$X_{370}$	$Y_{370}$	$X_{371}$	$Y_{371}$	$X_{372}$	$Y_{372}$	$X_{373}$	$Y_{373}$	$X_{374}$	$Y_{374}$	$X_{375}$	$Y_{375}$	$X_{376}$	$Y_{376}$	$X_{377}$	$Y_{377}$	$X_{378}$	$Y_{378}$	$X_{379}$	$Y_{379}$	$X_{380}$	$Y_{380}$	$X_{381}$	$Y_{381}$	$X_{382}$	$Y_{382}$	$X_{383}$	$Y_{383}$	$X_{384}$	$Y_{384}$	$X_{385}$	$Y_{385}$	$X_{386}$	$Y_{386}$	$X_{387}$	$Y_{387}$	$X_{388}$	$Y_{388}$	$X_{389}$	$Y_{389}$	$X_{390}$	$Y_{390}$	$X_{391}$	$Y_{391}$	$X_{392}$	$Y_{392}$	$X_{393}$	$Y_{393}$	$X_{394}$	$Y_{394}$	$X_{395}$	$Y_{395}$	$X_{396}$	$Y_{396}$	$X_{397}$	$Y_{397}$	$X_{398}$	$Y_{398}$	$X_{399}$	$Y_{399}$	$X_{400}$	$Y_{400}$	$X_{401}$	$Y_{401}$	$X_{402}$	$Y_{402}$	$X_{403}$	$Y_{403}$	$X_{404}$	$Y_{404}$	$X_{405}$	$Y_{405}$	$X_{406}$	$Y_{406}$	$X_{407}$	$Y_{407}$	$X_{408}$	$Y_{408}$	$X_{409}$	$Y_{409}$	$X_{410}$	$Y_{410}$	$X_{411}$	$Y_{411}$	$X_{412}$	$Y_{412}$	$X_{413}$	$Y_{413}$	$X_{414}$	$Y_{414}$	$X_{415}$	$Y_{415}$	$X_{416}$	$Y_{416}$	$X_{417}$	$Y_{417}$	$X_{418}$	$Y_{418}$	$X_{419}$	$Y_{419}$	$X_{420}$	$Y_{420}$	$X_{421}$	$Y_{421}$	$X_{422}$	$Y_{422}$	$X_{423}$	$Y_{423}$	$X_{424}$	$Y_{424}$	$X_{425}$	$Y_{425}$	$X_{426}$	$Y_{426}$	$X_{427}$	$Y_{427}$	$X_{428}$	$Y_{428}$	$X_{429}$	$Y_{429}$	$X_{430}$	$Y_{430}$	$X_{431}$	$Y_{431}$	$X_{432}$	$Y_{432}$	$X_{433}$	$Y_{433}$	$X_{434}$	$Y_{434}$	$X_{435}$	$Y_{435}$	$X_{436}$	$Y_{436}$	$X_{437}$	$Y_{437}$	$X_{438}$	$Y_{438}$	$X_{439}$	$Y_{439}$	$X_{440}$	$Y_{440}$	$X_{441}$	$Y_{441}$	$X_{442}$	$Y_{442}$	$X_{443}$	$Y_{443}$	$X_{444}$	$Y_{444}$	$X_{445}$	$Y_{445}$	$X_{446}$	$Y_{446}$	$X_{447}$	$Y_{447}$	$X_{448}$	$Y_{448}$	$X_{449}$	$Y_{449}$	$X_{450}$	$Y_{450}$	$X_{451}$	$Y_{451}$	$X_{452}$	$Y_{452}$	$X_{453}$	$Y_{453}$	$X_{454}$	$Y_{454}$	$X_{455}$	$Y_{455}$	$X_{456}$	$Y_{456}$	$X_{457}$	$Y_{457}$	$X_{458}$	$Y_{458}$	$X_{459}$	$Y_{459}$	$X_{460}$	$Y_{460}$	$X_{461}$	$Y_{461}$	$X_{462}$	$Y_{462}$	$X_{463}$	$Y_{463}$	$X_{464}$	$Y_{464}$	$X_{465}$	$Y_{465}$	$X_{466}$	$Y_{466}$	$X_{467}$	$Y_{467}$	$X_{468}$	$Y_{468}$	$X_{469}$	$Y_{469}$	$X_{470}$	$Y_{470}$	$X_{471}$	$Y_{471}$	$X_{472}$	$Y_{472}$	$X_{473}$	$Y_{473}$	$X_{474}$	$Y_{474}$	$X_{475}$	$Y_{475}$	$X_{476}$	$Y_{476}$	$X_{477}$	$Y_{477}$	$X_{478}$	$Y_{478}$	$X_{479}$	$Y_{479}$	$X_{480}$	$Y_{480}$	$X_{481}$	$Y_{481}$	$X_{482}$	$Y_{482}$	$X_{483}$	$Y_{483}$	$X_{484}$	$Y_{484}$	$X_{485}$	$Y_{485}$	$X_{486}$	$Y_{486}$	$X_{487}$	$Y_{487}$	$X_{488}$	$Y_{488}$	$X_{489}$	$Y_{489}$	$X_{490}$	$Y_{490}$	$X_{491}$	$Y_{491}$	$X_{492}$	$Y_{492}$	$X_{493}$	$Y_{493}$	$X_{494}$	$Y_{494}$	$X_{495}$	$Y_{495}$	$X_{496}$	$Y_{496}$	$X_{497}$	$Y_{497}$	$X_{498}$	$Y_{498}$	$X_{499}$	$Y_{499}$	$X_{500}$	$Y_{500}$	$X_{501}$	$Y_{501}$	$X_{502}$	$Y_{502}$	$X_{503}$	$Y_{503}$	$X_{504}$	$Y_{504}$	$X_{505}$	$Y_{505}$	$X_{506}$	$Y_{506}$	$X_{507}$	$Y_{507}$	$X_{508}$	$Y_{508}$	$X_{509}$	$Y_{509}$	$X_{510}$	$Y_{510}$	$X_{511}$	$Y_{511}$	$X_{512}$	$Y_{512}$	$X_{513}$	$Y_{513}$	$X_{514}$	$Y_{514}$	$X_{515}$	$Y_{515}$	$X_{516}$	$Y_{516}$	$X_{517}$	$Y_{517}$	$X_{518}$	$Y_{518}$	$X_{519}$	$Y_{519}$	$X_{520}$	$Y_{520}$	$X_{521}$	$Y_{521}$	$X_{522}$	$Y_{522}$	$X_{523}$	$Y_{523}$	$X_{524}$	$Y_{524}$	$X_{525}$	$Y_{525}$	$X_{526}$	$Y_{526}$	$X_{527}$	$Y_{527}$	$X_{528}$	$Y_{528}$	$X_{529}$	$Y_{529}$	$X_{530}$	$Y_{530}$	$X_{531}$	$Y_{531}$	$X_{532}$	$Y_{532}$	$X_{533}$	$Y_{533}$	$X_{534}$	$Y_{534}$	$X_{535}$	$Y_{535}$	$X_{536}$	$Y_{536}$	$X_{537}$	$Y_{537}$	$X_{538}$	$Y_{538}$	$X_{539}$	$Y_{539}$	$X_{540}$	$Y_{540}$	$X_{541}$	$Y_{541}$	$X_{542}$	$Y_{542}$	$X_{543}$	$Y_{543}$	$X_{544}$	$Y_{544}$	$X_{545}$	$Y_{545}$	$X_{546}$	$Y_{546}$	$X_{547}$	$Y_{547}$	$X_{548}$	$Y_{548}$	$X_{549}$	$Y_{549}$	$X_{550}$	$Y_{550}$	$X_{551}$	$Y_{551}$	$X_{552}$	$Y_{552}$	$X_{553}$	$Y_{553}$	$X_{554}$	$Y_{554}$	$X_{555}$	$Y_{555}$	$X_{556}$	$Y_{556}$	$X_{557}$	$Y_{557}$	$X_{558}$	$Y_{558}$	$X_{559}$	$Y_{559}$	$X_{560}$	$Y_{560}$	$X_{561}$	$Y_{561}$	$X_{562}$	$Y_{562}$	$X_{563}$	$Y_{563}$	$X_{564}$	$Y_{564}$	$X_{565}$	$Y_{565}$	$X_{566}$	$Y_{566}$	$X_{567}$	$Y_{567}$	$X_{568}$	$Y_{568}$	$X_{569}$	$Y_{569}$	$X_{570}$	$Y_{570}$	$X_{571}$	$Y_{571}$	$X_{572}$	$Y_{572}$	$X_{573}$	$Y_{573}$	$X_{574}$	$Y_{574}$	$X_{575}$	$Y_{575}$	$X_{576}$	$Y_{576}$	$X_{577}$	$Y_{577}$	$X_{578}$	$Y_{578}$	$X_{579}$	$Y_{579}$	$X_{580}$	$Y_{580}$	$X_{581}$	$Y_{581}$	$X_{582}$	$Y_{582}$	$X_{583}$	$Y_{583}$	$X_{584}$	$Y_{584}$	$X_{585}$	$Y_{585}$	$X_{586}$	$Y_{586}$	$X_{587}$	$Y_{587}$	$X_{588}$	$Y_{588}$	$X_{589}$	$Y_{589}$	$X_{590}$	$Y_{590}$	$X_{591}$	$Y_{591}$	$X_{592}$	$Y_{592}$	$X_{593}$	$Y_{593}$	$X_{594}$	$Y_{594}$	$X_{595}$	$Y_{595}$	$X_{596}$	$Y_{596}$	$X_{597}$	$Y_{597}$	$X_{598}$	$Y_{598}$	$X_{599}$	$Y_{599}$	$X_{600}$	$Y_{600}$	$X_{601}$	$Y_{601}$	$X_{602}$	$Y_{602}$	$X_{603}$	$Y_{603}$	$X_{604}$	$Y_{604}$	$X_{605}$	$Y_{605}$	$X_{606}$	$Y_{606}$	$X_{607}$	$Y_{607}$	$X_{608}$	$Y_{608}$	$X_{609}$	$Y_{609}$	$X_{610}$	$Y_{610}$	$X_{611}$	$Y_{611}$	$X_{612}$	$Y_{612}$	$X_{613}$	$Y_{613}$	$X_{614}$	$Y_{614}$	$X_{615}$	$Y_{615}$	$X_{616}$	$Y_{616}$	$X_{617}$	$Y_{617}$	$X_{618}$	$Y_{618}$	$X_{619}$	$Y_{619}$	$X_{620}$	$Y_{620}$	$X_{621}$	$Y_{621}$	$X_{622}$	$Y_{622}$	$X_{623}$	$Y_{623}$	$X_{624}$	$Y_{624}$	$X_{625}$	$Y_{625}$
1	0	1	1	1	0	1	0	1	0	0	0	1	0	0	0	1	0	1	0	0	0	0	0	0	1	1	0	1	0	0	0	1	0	1	1	1	0	1	1	1	0	0	1	1	0	1	1	0	1	0	1	1	0	0	0	0	0	1	0	1	1	0	0	1	0	0	1	1	0	1	0	1	1	1	0	1	0	1	1	1	0	1	0	0	1	0	0	1	1	1	0	1	0	0	0	1	1	1	0	0	0	1	1	0	1	1	0	1	0	1	0	1	0	1	1	0	1	1	0	1	1	1	0	1	1	0	1	1	0	0	0	0	0	1	0	1	0	0	0	1	1	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	1	1	0	1	0	1	0	1	1	1	1	1	0	1	0	1	0	1	0	1	1	1	0	1	0	0	0	1	0	1	0	1	1	0	0	1	0	0	0	1	0	1	0	1	0	1	1	1	0	1	0	1	1	0	0	1	0	1	1	0	1	1	1	1	0	0	1	0	1	1	0	0	0	1	0	0	0	1	0	1	0	1	0	1	0	0	0	1	0	0	1	1	1	0	0	0	0	1	0	0	0	0	0	0	0	1	0	1	0	1	0	1	1	1	0	0	1	1	1	0	0	0	0	0	0	0	0	1	0	1	0	1	0	1	0	1	0	1	0	0	0	1	0	1	0	1	0	0	0	0	0	0	1	1	0	1	1	1	0	1	0	1	0	1	0	1	1	1	0	0	0	0	0	1	0	0	0	1	0	1	0	0	0	0	0	1	1	1	0	0	1	1	1	0	0	1	0	0	0	0	0	0	0	1	0	1	0	1	0	1	0	1	1	1	0	1	0	0	1	1	0	1	0	1	0	1	1	1	1	0	0	1	0	1	1	0	0	1	0	0	1	1	0	1	1	0	0	1	0	1	0	1	0	1	0	1	0	1	1	1	1	1	0	1	0	0	0	1	0	1	1	0	0	1	0	1	0	1	0	0	0	1	0	1	0	0	0	1	0	0	0	0	0	1	0	0	0	1	1	1	0	1	0	1	0	1	0	1	0	0	0	0	0	0	1	1	1	1	1	1	0	1	0	0	0	1	1	1	0	1	0	1	0	0	1	1	1	1	0	1	0	1	0	1	0	1	0	1	0	0	0	1	0	1	1	1	1	1	1	1	0	0	0	0	0	1	0	0	0	0	0	1	0	1	0	1	0	0	0	1	1	1	0	1	0	1	1	0	0	0	1	1	1	1	0	0	1	1	0	1	0	1	0	0	0	1	0	1	0	1	1	0	0	0	0	1	0	1	0	1	1	0	0	1	0	1	0	1	0	0	0	1	0	1	0	1	0	1	0	1	0	1	0	0	0	1	1	1	0	0	0	0	0	0	1	0	0	1	0	1	1	0	0	0	0	1	0	1	0	1	0	1	1	1	0	1	0	1	0	0	0	1	0	1	0	1	1	1	0	1	1	1	0	0	1	0	0	0	0	1	0	1	1	1	0	1	1	1	0	1	0	0	0	0	0	1	0	1	0	1	0	1	0	1	1	1	0	1	1	0	1	0	0	1	0	0	1	1	0	1	0	1	0	0	0	1	0	1	0	1	0	1	0	0	0	1	1	0	0	1	1	1	1	1	1	0	0	1	0	1	1	1	1	0	0	1	0	1	0	0	0	1	1	1	0	1	0	1	1	1	1	1	1	1	0	1	0	1	0	1	0	1	1	1	1	0	0	1	0	0	1	1	0	1	0	1	0	1	0	0	0	1	0	1	1	1	0	0	1	0	0	0	0	0	0	1	0	1	1	1	0	1	1	1	0	0	0	1	1	1	0	0	1	1	1	1	0	0	1	0	0	1	0	1	0	1	1	0	0	1	1	1	1	0	0	0	0	1	1	1	0	1	0	1	1	1	0	0	0	1	0	1	0	1	0	1	0	0	1	1	0	1	1	0	1	1	1	1	0	0	0	1	0	0	0	1	0	1	0	1	1	0	1	1	1	0	1	1	0	1	1	1	0	0	0	0	1	1	1	0	0	0	0	1	0	1	1	1	0	0	0	0	0	0	0	1	0	1	0	0	0	0	0	1	0	0	1	0	0	1	0	1	0	0	0	1	0	1	0	0	0	1	0	1	0	0	0	1	1	1	1	1	1	1	0	1	0	1	0	0	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	0	1	1	0	1	0	0	1	1	1	1	1	0	1	1	0	0	0	1	0	0	1	0	0	0	0	1	0	1	0	1	0	1	0	1	1	0	0	0	0	0	0	1	0	0	0	1	0	1	0	1	0	0	0	0	1	1	0	1	0	0	0	1	0	1	0	1	0	0	0	1	0	1	0	1	0	0	1	1	1	1	1	1	0	1	1	0	0	0	0	1	0	1	0	1	0	0	1	1	0	0	0	1	0	1	0	0	0	1	0	0	0	0	0	0	0	1	0	1	0	0	1	0	0	0	1	1	0	0	0	1	1	1	0	0	0	1	1	0	0	1	0	1	0	1	0	0	0	0	1	1	0	0	0	1	0	1	0	1	1	1	0	1	0	0	0	1	0	1	1	1	1	1	0	1	0	0	0	1	1	0	0	1	1	0	0	1	0	1	0	0	0	1	0	0	1	0	1	0	0	0	1	1	1	1	1	1	0	1	1	1	0	0	0	1	0	0	0	0	1	0	1	1	0	0	1	1	0	1	1	1	1	0	1	1	0	1	1	1	1	1	1	0	1	0	1	1	0	1	1	1	1	1	0	1	1	0	0	1	0	0	0	0	1	1	0	1	0	1
2	1	0	0	1	0	1	1	0	0	1	1	1	1	1	1	0	0	1	0	0	0	1	0	1	0	0	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	0	1	1	0	1	0	1	0	1	0	1	0	0	0	0	0	1	0	0	0	1	0	1	0	0	1	1	0	1	1	1	0	1	1	1	0	1	0	0	0	1	0	1	1	0	0	1	1	1	0	1	1	1	0	0	1	1	0	1	0	1	0	0	0	0	0	1	0	1	1	1	1	0	0	1	0	1	1	0	1	0	1	1	0	1	0	1	0	0	0	1	0	1	1	0	0	1	0	0	0	1	0	0	1	1	1	1	1	1	1	1	0	1	0	1	1	1	1	1	0	1	0	0	0	0	1	1	1	1	0	1	0	1	0	0	0	1	0	1	0	0	0	1	0	0	0	1	0	1	1	1	0	1	1	0	1	0	0	1	0	1	1	1	1	1	0	0	0	1	0	0	0	0	0	1	0	1	0	1	1	0	0	0	0	1	1	1	0	0	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	1	1	1	1	0	0	0	1	1	0	1	1	0	1	0	1	0	1	0	0	1	1	1	0	0	1	0	1	0	0	0	1	1	1	0	1	0	1	0	0	0	1	1	1	0	1	0	1	0	1	0	1	0	0	0	1	0	1	0	1	1	0	0	0	1	1	0	1	0	0	1	1	0	1	0	0	0	1	0	1	0	1	0	1	0	1	0	1	1	0	0	0	0	1	0	1	1	1	0	1	0	1	1	1	1	1	0	0	1	1	1	1	1	1	1	0	0	1	0	1	0	1	0	1	0	0	1	1	0	1	0	1	0	0	1	0	0	1	0	1	0	0	0	0	0	1	1	1	1	1	0	0	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	0	0	1	0	1	0	1	1	1	1	1	1	1	0	0	1	0	0	0	0	1	0	1	0	1	0	1	0	1	0	0	1	0	1	0	1	1	0	1	0	1	0	0	0	1	0	0	1	1	0	0	0	1	0	1	1	1	0	1	0	1	0	1	1	1	0	1	1	0	0	0	1	1	0	1	0	1	0	0	1	1	1	0	1	1	0	1	1	0	0	1	0	1	1	0	0	1	0	0	1	1	0	1	0	1	0	1	0	1	1	1	1	1	1	1	0	1	0	1	0	0	0	0	1	0	0	1	0	0	0	1	0	1	1	1	1	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	1	1	1	1	0	0	0	1	1	1	1	1	0	0	1	0	0	1	0	1	0	1	0	1	0	0	0	1	1	1	1	1	0	1	0	1	1	0	1	1	0	0	0	0	1	1	0	1	0	0	0	0	0	0	0	1	0	1	0	0	0	1	1	0	0	1	1	0	1	1	0	1	1	0	1	0	1	0	0	0	1	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	0	0	1	0	1	0	1	0	1	1	1	0	0	0	1	1	1	1	1	0	1	0	1	1	1	1	1	0	1	0	0	0	0	1	1	0	1	1	0	0	1	0	1	0	1	0	1	0	1	1	1	0	0	0	1	0	1	0	0	0	1	0	0	0	0	0	1	0	0	0	1	1	1	0	1	0	1	0	0	1	0	0	1	0	1	0	1	1	1	0	0	0	1	1	1	0	1	0	1	0	1	0	1	0	0	0	1	0	0	1	0	0	0	0	1	0	1	0	1	0	0	0	1	0	0	0	1	0	1	1	0	1	1	0	1	1	1	0	0	0	0	0	0	0	0	0	0	0	1	0	1	0	1	1	1	1	1	1	1	0	1	0	1	0	0	1	1	0	1	1	1	0	0	0	0	1	0	1	0	0	0	0	1	0	1	0	1	1	0	0	1	0	1	1	0	1	1	1	0	0	1	0	0	1	0	0	1	1	1	0	1	1	1	1	1	0	1	0	1	1	1	1	1	0	1	1	1	0	0	1	1	0	1	1	1	1	1	0	1	1	1	1	0	1	0	0	1	1	1	0	1	0	1	0	0	1	1	1	1	1	1	0	0	0	1	1	1	0	0	0	1	0	1	0	1	0	1	1	1	1	0	0	0	0	1	0	1	0	1	1	0	0	1	0	1	0	0	0	0	0	1	0	1	0	1	0	1	0	1	0	1	1	0	1	1	0	1	0	1	0	0	1	1	0	1	0	0	0	1	0	1	1	1	1	1	1	1	0	1	0	0	0	0	0	1	1	0	0	1	0	1	0	1	1	0	1	1	0	1	0	1	1	1	1	1	1	0	1	1	0	1	0	1	1	1	0	1	0	0	0	0	0	1	0	0	0	1	0	0	0	1	1	1	0	1	0	0	0	0	0	1	1	1	0	0	0	1	1	0	1	0	0	1	1	1	0	0	0	0	1	1	0	0	0	1	0	0	0	1	0	1	1	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	1	1	0	1	1	0	1	1	0	1	0	1	0	1	0	0	1	1	0	0	0	1	1	1	0	0	0	0	0	1	0	1	0	1	0	1	0	1	0	1	0	0	1	1	0	1	0	0	1	1	0	0	0	1	0	1	0	1	0	1	1	0	0	1	0	0	0	1	0	0	0	0	0	1	0	1	1	1	1	1	1	1	0	1	1	1	0	1	0	1	0	0	0	0	0	1	0	1	0	1	0	0	0	1	0	1	0	1	0	1	0	1	1	1	0	0	1	1	0	1	0	1	0	1	1	1	1	1	0	0	0	1
3	1	1	0	1	0	0	0	1	1	1	0	0	1	1	0	1	0	0	0	1	0	0	0	1	0	1	0	0	0	0	0	1	0	1	1	1	0	1	0	1	0	1	0	1	0	1	0	0	1	1	0	0	0	1	0	1	0	1	0	1	0	1	1	1	0	1	0	1	0	1	0	0	1	0	1	0	0	1	0	1	1	0	0	1	1	0	0	0	0	0	0	1	0	0	0	1	0	0	0	1	1	0	0	0	0	0	0	1	0	1	1	1	0	0	1	1	0	1	0	1	0	1	1	1	0	1	0	1	1	1	0	0	0	1	1	1	1	0	1	1	0	1	0	1	0	1	0	1	0	1	0	0	0	1	0	1	0	1	0	0	0	1	0	0	1	1	0	0	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	0	0	1	0	0	0	1	0	0	1	1	0	0	0	1	1	1	0	1	1	1	0	1	0	1	0	0	0	1	0	1	1	1	1	1	0	1	1	1	0	0	1	1	1	1	0	0	0	1	0	0	0	1	1	1	0	1	0	1	0	0	0	1	1	1	0	0	1	1	1	1	1	1	1	1	0	0	0	1	0	1	0	0	1	1	0	1	0	1	0	1	0	0	0	1	0	1	1	0	0	1	0	0	0	1	1	1	1	1	0	1	0	0	1	1	0	1	1	1	1	1	0	1	1	1	0	0	0	0	0	1	1	0	1	1	0	1	0	1	1	1	0	0	0	0	1	1	0	1	0	0	1	1	1	0	0	1	0	1	0	0	0	1	0	0	0	0	0	0	1	0	1	1	0	1	0	0	0	1	1	1	0	1	0	1	0	0	0	1	0	1	1	1	0	1	0	1	1	0	0	1	1	1	1	1	0	1	0	1	1	1	0	0	1	0	0	1	1	0	0	1	0	1	0	1	0	1	0	1	0	1	1	0	0	0	0	1	0	0	0	0	1	1	0	1	0	0	0	0	0	1	0	1	1	0	0	1	0	1	0	1	0	0	0	0	1	1	0	0	1	0	0	0	0	1	0	0	0	1	0	0	0	0	0	1	0	1	1	1	1	1	0	1	0	0	0	1	0	0	0	1	0	1	1	1	0	1	0	1	1	0	0	0	1	1	0	1	0	1	0	1	0	1	0	0	1	0	0	1	0	1	0	1	1	1	0	1	0	1	1	1	0	0	0	0	1	1	0	1	0	0	1	1	1	0	0	1	0	1	0	1	0	1	0	0	1	0	0	0	0	0	0	0	0	1	0	1	1	1	0	1	0	1	0	0	0	1	1	1	1	0	0	1	0	0	0	1	0	0	0	1	0	1	1	1	0	1	0	0	0	1	0	1	1	1	1	0	1	1	0	0	0	1	1	1	1	1	0	0	0	0	0	1	1	1	1	0	0	0	0	0	0	0	0	0	0	1	0	1	0	0	1	1	1	1	1	1	1	1	0	1	1	1	0	1	0	1	0	1	0	0	0	1	1	1	0	1	0	1	1	1	0	0	1	1	0	1	0	1	0	1	1	1	0	1	1	0	1	1	1	1	0	1	0	1	0	1	0	0	1	1	0	1	1	0	0	1	0	1	0	1	0	1	0	0	0	1	0	1	0	0	0	1	1	0	0	0	1	1	0	0	0	0	1	1	0	1	0	1	0	1	0	0	0	0	0	1	0	0	0	0	0	1	0	0	0	1	0	1	0	1	1	0	0	1	0	1	0	1	0	0	0	1	0	1	1	1	0	0	0	1	1	0	0	1	0	1	0	0	0	1	1	0	0	1	0	0	1	1	0	1	1	1	0	0	1	1	0	1	0	0	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	1	1	1	1	0	1	0	1	0	1	0	0	0	1	0	0	1	1	0	1	1	1	0	0	0	1	0	0	0	1	0	1	0	1	0	0	0	1	0	0	0	1	0	0	0	1	0	1	0	1	0	1	1	1	0	0	1	1	0	1	0	1	0	0	0	0	0	1	0	1	0	0	0	0	1	1	0	1	1	0	0	0	0	1	1	1	1	1	0	1	0	1	0	1	0	1	1	1	0	1	1	0	0	0	0	1	0	0	0	0	0	0	0	1	0	1	0	1	1	1	0	1	1	1	1	1	1	1	0	1	1	1	0	1	0	0	1	0	0	1	0	1	0	0	0	1	0	1	1	0	0	0	0	1	0	1	0	1	1	0	1	1	0	1	1	1	1	1	0	1	0	1	0	1	0	0	0	1	0	0	1	1	0	1	0	1	0	1	0	1	1	1	0	1	1	0	1	1	0	1	0	0	0	1	0	1	1	1	0	1	0	1	1	1	1	1	0	1	0	1	0	1	0	0	0	0	0	0	0	1	0	1	0	1	1	1	1	1	1	0	0	1	0	0	1	1	1	1	0	1	0	1	0	1	0	0	0	1	0	0	0	1	0	1	0	1	1	1	1	1	1	1	0	1	1	1	0	1	1	0	0	1	0	1	1	1	0	0	1	0	0	1	0	1	1	1	0	1	0	0	0	1	0	1	0	0	0	1	0	1	0	1	0	1	0	0	0	0	1	1	0	1	0	1	1	1	0	1	1	1	0	1	0	0	0	1	0	1	0	1	1	1	0	1	0	0	0	0	0	1	1	0	0	0	1	1	0	1	0	0	0	1	0	0	0	1	1	1	1	1	0	0	0	1	0	1	0	1	0	0	0	0	0	0	0	1	0	1	1	1	0	1	1	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	0	0	1	0	1	0	1	0	1
1M	0	1	0	1	0	1	1	1	0	1	0	1	1	1	0	1	0	1	1	0	0	1	1	0	0	0	0	1	0	0	0	0	1	1	0	1	0	1	0	0	0	1	0	1	0	1	1	1	1	0	0	1	0	1	0	1	1	0	0	0	0	1	1	1	0	0	1	1	0	1	1	1	1	1	1	1	1	1	0	0	0	0	0	0	0	1	0	1	0	1	0	1	0	0	0	1	0	1	0	0	1	1	1	1	0	0	0	1	1	1	0	1	1	1	0	1	0	1	0	1	0	1	1	1	1	0	1	1	1	1	0	1	0	1	0	1	0	1	1	1	0	1	1	1	0	0	0	0	0	1	1	1	0	1	1	1	0	1	1	0	1	0	0	0	0	1	0	1	1	1	0	0	0	0	0	1	0	1	0	0	0	0	1	1	0	1	1	1	0	1	1	1	0	1	0	1	1	1	0	0	0	1	1	1	0	1	1	1	0	1	0	1	0	1	1	1	0	0	0	1	1	1	0	1	0	0	1	0	0	1	0	1	1	1	0	1	0	0	0	0	0	1	0	1	0	1	0	1	0	1	1	1	0	0	0	1	0	1	0	1	0	1	1	1	0	1	0	1	0	1	1	1	0	1	0	0	1	1	1	1	1	1	0	1	1	1	1	1	1	1	1	1	0	1	0	1	0	1	0	0	0	0	0	1	0	1	0	1	0	1	0	1	0	0	0	0	0	0	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	1	1	0	0	0	0	1	1	0	1	0	1	0	1	1	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	1	1	1	0	0	1	0	1	0	1	0	0	1	1	0	1	1	1	1	1	0	0	0	1	0	0	0	0	1	0	0	1	1	0	1	1	1	1	1	0	0	1	0	0	0	0	0	1	0	1	1	0	1	0	0	1	1	1	0	1	0	1	1	1	1	0	0	0	0	0	0	1	0	1	0	1	0	0	1	1	0	0	1	1	1	1	1	1	1	1	0	1	0	1	0	0	0	1	0	1	0	1	1	1	0	0	0	1	0	1	1	1	0	1	0	0	0	1	0	1	0	0	1	1	0	1	0	0	1	1	1	1	0	1	0	1	0	1	1	1	0	1	0	1	1	1	1	1	0	1	1	1	0	0	0	1	1	1	1	1	0	1	1	1	1	1	0	1	0	1	1	0	0	1	0	0	0	0	1	1	1	1	0	1	1	1	1	0	0	0	0	1	0	1	0	1	0	1	1	1	0	0	1	1	0	1	0	0	1	0	0	1	0	0	0	0	0	0	1	1	0	0	1	1	0	0	0	1	0	0	0	0	1	1	1	1	0	0	1	0	0	1	0	0	0	1	1	1	0	1	1	1	0	0	0	0	0	1	0	0	1	1	1	1	1	1	0	1	1	1	0	1	1	1	0	1	0	1	0	1	0	1	1	1	0	1	0	1	0	1	0	1	0	0	0	1	0	0	1	1	1	1	0	1	0	1	0	1	0	1	0	1	0	1	0	0	1	1	0	0	0	1	1	1	0	1	0	0	0	0	0	0	1	1	0	1	0	0	0	1	0	1	0	1	0	0	0	0	1	0	1	1	0	1	0	1	0	1	1	1	1	1	1	1	0	1	0	1	1	0	1	1	0	1	0	1	0	0	0	1	0	0	0	1	0	1	0	1	0	0	0	1	0	1	0	1	0	1	0	1	1	1	0	1	0	1	0	1	1	1	0	1	1	1	0	0	0	1	0	0	0	0	0	0	1	0	0	1	0	0	0	1	0	0	1	1	1	1	0	1	0	1	0	1	0	1	0	1	1	1	0	0	0	0	0	1	0	0	1	1	0	0	1	1	0	0	0	0	0	1	1	1	1	0	1	1	1	1	0	0	0	1	0	0	0	0	1	1	0	1	0	0	1	1	0	1	1	0	0	0	0	0	0	1	0	0	1	0	1	1	0	1	0	1	1	1	0	1	0	1	0	0	0	1	0	1	0	1	0	0	0	0	1	1	0	0	0	1	0	0	0	1	1	1	1	0	0	0	0	1	0	1	0	1	0	0	0	0	1	0	1	1	1	1	0	0	0	0	1	1	0	1	0	1	0	1	0	0	0	1	1	0	0	1	0	0	0	0	0	0	0	1	0	1	0	0	0	1	1	1	0	1	0	1	1	1	0	0	0	0	1	1	0	0	1	1	1	0	0	0	1	1	0	1	1	1	0	1	1	1	1	1	0	1	0	1	0	0	0	0	0	1	0	1	1	1	0	1	0	1	0	1	0	0	0	1	0	1	0	0	0	1	0	1	1	1	0	1	0	1	0	0	0	1	1	1	1	1	0	1	0	0	0	1	0	1	0	1	0	1	1	0	1	0	0	0	0	1	1	1	0	0	0	1	1	1	0	1	0	1	0	0	1	1	1	1	0	1	0	1	0	0	0	1	0	1	0	1	1	1	0	1	0	1	0	1	0	1	0	1	0	0	0	1	0	1	1	1	0	1	0	0	1	1	0	1	1	0	0	1	0	1	1	0	0	1	0	1	0	1	0	0	0	1	0	0	1	0	0	1	0	1	1	1	0	0	0	1	0	1	0	1	0	0	0	1	0	0	0	1	0	0	1	1	0	1	1	1	0	1	1	1	1	1	1	1	1	1	0	0	1	0	0	1	0	0	0	1	1	0	1	0	1	1	0	1	1	0	0	1	1	1	1	1	0	1	0	1	1	1	0	0	0	0	0	0	1	0	0	1	0	1	0	0	0	0	0	1	0	1	0	1	1	1	0	1	1	1	0	1

::: ::: {.column width=40%}

summaries
$\bar X$	$\bar{Y}$	$\frac{\sum_{i:X_i=0}Y_i}{\sum_{i:X_i=0}1}$	$\frac{\sum_{i:X_i=1}Y_i}{\sum_{i:X_i=1}1}$	difference
0.28	0.68	0.68	0.69	0.01
0.29	0.71	0.72	0.70	-0.01
0.26	0.70	0.68	0.76	0.08
0.30	0.71	0.68	0.79	0.12

::: ::::

Junk?

$$ { \[\begin{array}{r|rr|rr|r|rr|rrrr} \text{call} & 1 & & 2 & & \dots & 625 & & & & & & \\ \text{poll} & X_1 & Y_1 & X_2 & Y_2 & \dots & X_{625} & Y_{625} & \overline{X} & \overline{Y} &\frac{\sum_{i:X_i=0} Y_i}{\sum_{i:X_i=0} 1} & \frac{\sum_{i:X_i=1} Y_i}{\sum_{i:X_i=1} 1} & \text{difference} \\ \hline \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}0 & \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}\dots & \color[RGB]{7,59,76}0 & \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}0.28 & \color[RGB]{7,59,76}0.68 & \color[RGB]{7,59,76}0.68 & \color[RGB]{7,59,76}0.69 & \color[RGB]{7,59,76}0.01 \\ \color[RGB]{239,71,111}1 & \color[RGB]{239,71,111}1 & \color[RGB]{239,71,111}0 & \color[RGB]{239,71,111}0 & \color[RGB]{239,71,111}1 & \color[RGB]{239,71,111}\dots & \color[RGB]{239,71,111}0 & \color[RGB]{239,71,111}1 & \color[RGB]{239,71,111}0.29 & \color[RGB]{239,71,111}0.71 & \color[RGB]{239,71,111}0.72 & \color[RGB]{239,71,111}0.70 & \color[RGB]{239,71,111}-0.01 \\ \color[RGB]{17,138,178}1 & \color[RGB]{17,138,178}1 & \color[RGB]{17,138,178}1 & \color[RGB]{17,138,178}0 & \color[RGB]{17,138,178}1 & \color[RGB]{17,138,178}\dots & \color[RGB]{17,138,178}0 & \color[RGB]{17,138,178}1 & \color[RGB]{17,138,178}0.26 & \color[RGB]{17,138,178}0.70 & \color[RGB]{17,138,178}0.68 & \color[RGB]{17,138,178}0.76 & \color[RGB]{17,138,178}0.08 \\ \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots \\ \color[RGB]{6,214,160}1 & \color[RGB]{6,214,160}0 & \color[RGB]{6,214,160}1 & \color[RGB]{6,214,160}0 & \color[RGB]{6,214,160}1 & \color[RGB]{6,214,160}\dots & \color[RGB]{6,214,160}0 & \color[RGB]{6,214,160}1 & \color[RGB]{6,214,160}0.30 & \color[RGB]{6,214,160}0.71 & \color[RGB]{6,214,160}0.68 & \color[RGB]{6,214,160}0.79 & \color[RGB]{6,214,160}0.12 \\ \end{array}\]

} $$

Bootstrapping The Difference

Our Sample

Three Bootstrap Samples—The First ‘Call’

Our Sample

Three Bootstrap Samples—The First ‘Call’

Our Sample

Three Bootstrap Samples—The First, Second, and Last ‘Call’

\[ \small{ \begin{array}{r|rr|rr|r|rr|rrrr} \text{call} & 1 & & 2 & & \dots & 625 & & & & & & \\ \text{poll} & X_1 & Y_1 & X_2 & Y_2 & \dots & X_{625} & Y_{625} & \overline{X} & \overline{Y} &\frac{\sum_{i:X_i=0} Y_i}{\sum_{i:X_i=0} 1} & \frac{\sum_{i:X_i=1} Y_i}{\sum_{i:X_i=1} 1} & \text{difference} \\ \hline \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}0 & \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}\dots & \color[RGB]{7,59,76}0 & \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}0.28 & \color[RGB]{7,59,76}0.68 & \color[RGB]{7,59,76}0.68 & 0.69 & \color[RGB]{7,59,76}0.01 \end{array} } \]

\[ \small{ \begin{array}{r|rr|rr|r|rr|rrrr} \text{`call'} & 1 & & 2 & & \dots & 625 & & & & & & \\ \text{`poll'} & X_1^* & Y_1^* & X_2^* & Y_2^* & \dots & X^*_{625} & Y^*_{625} & \overline{X}^* & \overline{Y}^* &\frac{\sum_{i:X_i^*=0} Y_i}{\sum_{i:X_i^*=0} 1} & \frac{\sum_{i:X_i^*=1} Y_i}{\sum_{i:X_i^*=1} 1} & \text{difference} \\ \hline \color[RGB]{239,71,111}1 & \color[RGB]{239,71,111}X_{398} & \color[RGB]{239,71,111}Y_{398} & & & \color[RGB]{239,71,111}\dots & & & & & & & & \\ & \color[RGB]{239,71,111}1 & \color[RGB]{239,71,111}0 & & & \color[RGB]{239,71,111}\dots & & & & & & & & \\ \color[RGB]{17,138,178}2 & \color[RGB]{17,138,178}X_{293} & \color[RGB]{17,138,178}Y_{293} & & & \color[RGB]{17,138,178}\dots & & & & & & & & \\ & \color[RGB]{17,138,178}0 & \color[RGB]{17,138,178}1 & & & \color[RGB]{17,138,178}\dots & & & & & & & & \\ \color[RGB]{6,214,160}3 & \color[RGB]{6,214,160}X_{281} & \color[RGB]{6,214,160}Y_{281} & & & \color[RGB]{6,214,160}\dots & & & & & & & & \\ & \color[RGB]{6,214,160}0 & \color[RGB]{6,214,160}0 & & & \color[RGB]{6,214,160}\dots & & & & & & & & \\ \end{array} } \]

A Table of Bootstrap Polls

Our Sample

Three Bootstrap Samples

\[ \small{ \begin{array}{r|rr|rr|r|rr|rrrr} \text{call} & 1 & & 2 & & \dots & 625 & & & & & & \\ \text{poll} & X_1 & Y_1 & X_2 & Y_2 & \dots & X_{625} & Y_{625} & \overline{X} & \overline{Y} &\frac{\sum_{i:X_i=0} Y_i}{\sum_{i:X_i=0} 1} & \frac{\sum_{i:X_i=1} Y_i}{\sum_{i:X_i=1} 0} & \text{difference} \\ \hline \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}0 & \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}\dots & \color[RGB]{7,59,76}0 & \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}0.28 & \color[RGB]{7,59,76}0.68 & \color[RGB]{7,59,76}0.68 & 0.69 & \color[RGB]{7,59,76}0.01 \end{array} } \]

\[ \small{ \begin{array}{r|rr|rr|r|rr|rrrr} \text{`call'} & 1 & & 2 & & \dots & 625 & & & & & & \\ \text{`poll'} & X_1^* & Y_1^* & X_2^* & Y_2^* & \dots & X^*_{625} & Y^*_{625} & \overline{X}^* & \overline{Y}^* &\frac{\sum_{i:X_i^*=0} Y_i}{\sum_{i:X_i^*=0} 1} & \frac{\sum_{i:X_i^*=1} Y_i}{\sum_{i:X_i^*=1} 1} & \text{difference} \\ \hline \color[RGB]{239,71,111}2 & \color[RGB]{239,71,111}X_{398} & \color[RGB]{239,71,111}Y_{398} & \color[RGB]{239,71,111}X_{129} & \color[RGB]{239,71,111}Y_{129} & \color[RGB]{239,71,111}\dots & \color[RGB]{239,71,111}X_{232} & \color[RGB]{239,71,111}Y_{232} & & & & & & \\ & \color[RGB]{239,71,111}1 & \color[RGB]{239,71,111}0 & \color[RGB]{239,71,111}1 & \color[RGB]{239,71,111}1 & \color[RGB]{239,71,111}\dots & \color[RGB]{239,71,111}0 & \color[RGB]{239,71,111}1 & \color[RGB]{239,71,111}0.29 & \color[RGB]{239,71,111}0.68 & \color[RGB]{239,71,111}0.69 & \color[RGB]{239,71,111}0.68 & \color[RGB]{239,71,111}-0.01 \\ \color[RGB]{17,138,178}2 & \color[RGB]{17,138,178}X_{293} & \color[RGB]{17,138,178}Y_{293} & \color[RGB]{17,138,178}X_{526} & \color[RGB]{17,138,178}Y_{526} & \color[RGB]{17,138,178}\dots & \color[RGB]{17,138,178}X_{578} & \color[RGB]{17,138,178}Y_{578} & & & & & & \\ & \color[RGB]{17,138,178}0 & \color[RGB]{17,138,178}1 & \color[RGB]{17,138,178}0 & \color[RGB]{17,138,178}1 & \color[RGB]{17,138,178}\dots & \color[RGB]{17,138,178}0 & \color[RGB]{17,138,178}1 & \color[RGB]{17,138,178}0.28 & \color[RGB]{17,138,178}0.65 & \color[RGB]{17,138,178}0.64 & \color[RGB]{17,138,178}0.67 & \color[RGB]{17,138,178}0.03 \\ \color[RGB]{6,214,160}1M & \color[RGB]{6,214,160}X_{281} & \color[RGB]{6,214,160}Y_{281} & \color[RGB]{6,214,160}X_{520} & \color[RGB]{6,214,160}Y_{520} & \color[RGB]{6,214,160}\dots & \color[RGB]{6,214,160}X_{363} & \color[RGB]{6,214,160}Y_{363} & & & & & & \\ & \color[RGB]{6,214,160}0 & \color[RGB]{6,214,160}0 & \color[RGB]{6,214,160}0 & \color[RGB]{6,214,160}1 & \color[RGB]{6,214,160}\dots & \color[RGB]{6,214,160}0 & \color[RGB]{6,214,160}1 & \color[RGB]{6,214,160}0.28 & \color[RGB]{6,214,160}0.68 & \color[RGB]{6,214,160}0.66 & \color[RGB]{6,214,160}0.71 & \color[RGB]{6,214,160}0.05 \\ \end{array} } \]

The Difference’s Bootstrap Sampling Distribution

difference.bootstrap.samples = array(dim=10000)
for(rr in 1:10000) {
    I = sample(1:n, n, replace=TRUE)
    Xstar = X[I]
    Ystar = Y[I]
    difference.bootstrap.samples[rr] = mean(Ystar[Xstar==1]) - mean(Ystar[Xstar==0])
}

Not bad. Our bootstrap sampling distribution looks a lot like our actual sampling distribution.
And, in particular, the intervals widths we calculate are very close: 0.156 vs. 0.163.
And that’s really all that matters. Q. Why?

Because that’s what we use to calibrate our interval estimate.
- It’s our point estimate — the black dot
- With arms wide enough to span 95% of the bootstrap sampling distribution — the red-shaded interval.

Why Does The Bootstrap Work?

Our Binomial-Specific Argument

Last class, we argued that we could estimate the sampling distribution of a proportion well enough.
Our argument used the parametric form of the sampling distribution: the binomial distribution.
- The width of the binomial probability distribution doesn’t change quickly as the probability of heads changes.
- So if we have an ok estimate of the probability of heads, i.e. the sample proportion $\bar Y$ …
- then we can use that instead of the actual probability of heads, i.e. the population proportion $\bar y$.

\[ \bar Y \approx \bar y \implies \text{Binomial}(n,\bar Y) \approx \text{Binomial}(n,\bar y) \]

This told us the bootstrap worked because the bootstrap was doing exactly this.
That is, the bootstrap sampling distribution is Binomial with heads probability $\bar Y$.
But the bootstrap still works. Why? Ultimately, it’s a subtle question with a lot of different answers.
I’m going to give you a simple one. One that’s very similar to the one we used last class.
Instead of equivalance to a parametric estimate, we’ll argue using approximation by one.

The Same Idea With Normal Approximation

Here I’ve drawn the same diagram as before, but with the binomial distribution replaced by a normal approximation to it. \[ f_{\mu,\sigma}(x) = \frac{1}{\sqrt{2\pi}\sigma} e^{-\frac{(x-\mu)^2}{2\sigma^2}} \qfor \mu=p \qand \sigma=\sqrt{p(1-p)/n} \]
This is a good approximation. I can’t tell the difference between this diagram and the last one.
And we can repeat our argument using the normal distribution’s parametric form.
- The width of the normal distribution doesn’t change too fast as its standard deviation $\sigma$ does.
- So if we have a good estimate of that standard deviation, e.g. $\sqrt{p(1-p)/n}$ where $p$ is the sample proportion $\bar Y$ …
- then we can use that instead of the actual standard deviation, which is the same where $p$ is the population proportion $\bar y$.
This argument generalizes well. It works, e.g., for the difference in proportions.
It works whenever the actual and boostrap sampling distributions …
- are both approximately normal
- and have similar standard deviations.

Approximate Normality for the Difference in Turnout

A histogram of draws from the actual sampling distribution and its normal approximation.

A histogram of draws from the bootstrap sampling distribution and its normal approximation.

The actual sampling distribution of the difference in turnout is approximately normal.
So is the bootstrap sampling distribution. And they do have similar standard deviations.
Most things involving averages are approximately normal. This is a consequence of the Central Limit Theorem.
We won’t prove that, so you’ll have to take my word for it.
- There’s a very nice proof that doesn’t really require any background you don’t have.
- It uses an argument called Stein’s method that’s very interesting and very widely applicable.
- But it’d take us a couple of lectures and we don’t have time for that.

The Same Idea for the Difference in Turnout

We can use almost the same picture to argue that the bootstrap works for the difference in turnout.
But there is a difference.
- The spread of a proportion’s sampling distribution is determined by that proportion.
- But the spread of the difference isn’t determined by the difference.
- It’s determined by the two proportions we’re comparing.
This means we can’t summarize the range of interval estimates we tend to get as we did before.
i.e., by drawing a single interval around the 2.5th, 50th, and 97.5th percentiles of our sampling distribution.
Instead, we need to draw intervals at a range of widths.
Above, I’ve drawn intervals three intervals around each point estimate.
One each at the 2.5th, 50th, and 97.5th percentiles of the sampling distribution of interval estimate widths.

Summary

Most of the time, the bootstrap sampling distribution is a good approximation of the actual sampling distribution.
This means we have a general procedure we can use to calibrating interval estimates: we use the bootstrap.
The downside is that it’s a bit opaque.
- We can use it to approximate our point estimate’s sampling distribution after we’ve run our experiment.
- But that doesn’t tell us what it will look like before we’ve run it.
That matters when we’re designing experiments. It’s important to know what we will and won’t be able to learn.
- e.g., it’s important to make our experiments big enough that we get the precision we need.
- This is called power analysis and it’s a big part of experimental design.
Next class, we’ll talk about how to use normal approximation to do that.

Lecture 4

Point and Interval Estimation

Calibration of Interval Estimates

A Parametric Estimate of the Sampling Distribution

A Parametric Estimate of the Sampling Distribution

The Bootstrap Interpretation

The Bootstrap Interpretation

Bootstrapping

The Bootstrap is Nonparametric

Comparing Black and Non-Black Turnout

A Covariate? What’s That?

Comparing Black and Non-Black Turnout

The Turnout Difference’s Sampling Distribution

Junk?

Bootstrapping The Difference

A Table of Bootstrap Polls

The Difference’s Bootstrap Sampling Distribution

Why Does The Bootstrap Work?

Our Binomial-Specific Argument

The Same Idea With Normal Approximation

Approximate Normality for the Difference in Turnout

The Same Idea for the Difference in Turnout

Summary